A physiologically-adapted gold standard for arousal during stress

Emotion is an inherently subjective psychophysiological human-state and to produce an agreed-upon representation (gold standard) for continuous emotion requires a time-consuming and costly training procedure of multiple human annotators. There is strong evidence in the literature that physiological signals are sufficient objective markers for states of emotion, particularly arousal. In this contribution, we utilise a dataset which includes continuous emotion and physiological signals - Heartbeats per Minute (BPM), Electrodermal Activity (EDA), and Respiration-rate - captured during a stress inducing scenario (Trier Social Stress Test). We utilise a Long Short-Term Memory, Recurrent Neural Network to explore the benefit of fusing these physiological signals with arousal as the target, learning from various audio, video, and textual based features. We utilise the state-of-the-art MuSe-Toolbox to consider both annotation delay and inter-rater agreement weighting when fusing the target signals. An improvement in Concordance Correlation Coefficient (CCC) is seen across features sets when fusing EDA with arousal, compared to the arousal only gold standard results. Additionally, BERT-based textual features' results improved for arousal plus all physiological signals, obtaining up to .3344 CCC compared to .2118 CCC for arousal only. Multimodal fusion also improves overall CCC with audio plus video features obtaining up to .6157 CCC to recognize arousal plus EDA and BPM

Towards Multimodal Prediction of Spontaneous Humour: A Novel Dataset and First Results

Humour is a substantial element of human affect and cognition. Its automatic understanding can facilitate a more naturalistic human-device interaction and the humanisation of artificial intelligence. Current methods of humour detection are solely based on staged data making them inadequate for 'real-world' applications. We address this deficiency by introducing the novel Passau-Spontaneous Football Coach Humour (Passau-SFCH) dataset, comprising of about 11 hours of recordings. The Passau-SFCH dataset is annotated for the presence of humour and its dimensions (sentiment and direction) as proposed in Martin's Humor Style Questionnaire. We conduct a series of experiments, employing pretrained Transformers, convolutional neural networks, and expert-designed features. The performance of each modality (text, audio, video) for spontaneous humour recognition is analysed and their complementarity is investigated. Our findings suggest that for the automatic analysis of humour and its sentiment, facial expressions are most promising, while humour direction can be best modelled via text-based features. The results reveal considerable differences among various subjects, highlighting the individuality of humour usage and style. Further, we observe that a decision-level fusion yields the best recognition result. Finally, we make our code publicly available at https://www.github.com/EIHW/passau-sfch. The Passau-SFCH dataset is available upon request.Comment: This work has been submitted to the IEEE for possible publication. Copyright may be transferred without notice, after which this version may no longer be accessible (Major Revision

The MuSe 2021 Multimodal Sentiment Analysis Challenge: sentiment, emotion, physiological-emotion, and stress

Multimodal Sentiment Analysis (MuSe) 2021 is a challenge focusing on the tasks of sentiment and emotion, as well as physiological-emotion and emotion-based stress recognition through more comprehensively integrating the audio-visual, language, and biological signal modalities. The purpose of MuSe 2021 is to bring together communities from different disciplines; mainly, the audio-visual emotion recognition community (signal-based), the sentiment analysis community (symbol-based), and the health informatics community. We present four distinct sub-challenges: MuSe-Wilder and MuSe-Stress which focus on continuous emotion (valence and arousal) prediction; MuSe-Sent, in which participants recognise five classes each for valence and arousal; and MuSe-Physio, in which the novel aspect of 'physiological-emotion' is to be predicted. For this year's challenge, we utilise the MuSe-CaR dataset focusing on user-generated reviews and introduce the Ulm-TSST dataset, which displays people in stressful depositions. This paper also provides detail on the state-of-the-art feature sets extracted from these datasets for utilisation by our baseline model, a Long Short-Term Memory-Recurrent Neural Network. For each sub-challenge, a competitive baseline for participants is set; namely, on test, we report a Concordance Correlation Coefficient (CCC) of .4616 CCC for MuSe-Wilder; .5088 CCC for MuSe-Stress, and .4908 CCC for MuSe-Physio. For MuSe-Sent an F1 score of 32.82% is obtained

The MuSe 2022 Multimodal Sentiment Analysis Challenge: Humor, Emotional Reactions, and Stress

The Multimodal Sentiment Analysis Challenge (MuSe) 2022 is dedicated to multimodal sentiment and emotion recognition. For this year's challenge, we feature three datasets: (i) the Passau Spontaneous Football Coach Humor (Passau-SFCH) dataset that contains audio-visual recordings of German football coaches, labelled for the presence of humour; (ii) the Hume-Reaction dataset in which reactions of individuals to emotional stimuli have been annotated with respect to seven emotional expression intensities, and (iii) the Ulm-Trier Social Stress Test (Ulm-TSST) dataset comprising of audio-visual data labelled with continuous emotion values (arousal and valence) of people in stressful dispositions. Using the introduced datasets, MuSe 2022 2022 addresses three contemporary affective computing problems: in the Humor Detection Sub-Challenge (MuSe-Humor), spontaneous humour has to be recognised; in the Emotional Reactions Sub-Challenge (MuSe-Reaction), seven fine-grained `in-the-wild' emotions have to be predicted; and in the Emotional Stress Sub-Challenge (MuSe-Stress), a continuous prediction of stressed emotion values is featured. The challenge is designed to attract different research communities, encouraging a fusion of their disciplines. Mainly, MuSe 2022 targets the communities of audio-visual emotion recognition, health informatics, and symbolic sentiment analysis. This baseline paper describes the datasets as well as the feature sets extracted from them. A recurrent neural network with LSTM cells is used to set competitive baseline results on the test partitions for each sub-challenge. We report an Area Under the Curve (AUC) of .8480 for MuSe-Humor; .2801 mean (from 7-classes) Pearson's Correlations Coefficient for MuSe-Reaction, as well as .4931 Concordance Correlation Coefficient (CCC) and .4761 for valence and arousal in MuSe-Stress, respectively.Comment: Preliminary baseline paper for the 3rd Multimodal Sentiment Analysis Challenge (MuSe) 2022, a full-day workshop at ACM Multimedia 202

Lateral one-third gland resection in Cushing patients with failed adenoma identification leads to low remission rates: long-term observations from a small, single-center cohort.

BACKGROUND Currently, there are no guidelines for neurosurgeons treating patients with Cushing's disease (CD) when intraoperative adenoma identification is negative. Under these circumstances, a total hypophysectomy or hemi-hypophysectomy on the side indicated by inferior petrosal sinus sampling (IPSS) is the approach being used, although there is a subsequent risk of hypopituitarism. Data on whether one-third lateral pituitary gland resection results in cure of hypercortisolism and low rates of hypopituitarism remain inconclusive. METHODS Retrospective single-center study of CD patients with failed intraoperative adenoma identification and subsequent resection of the lateral one-third of the pituitary gland as predicted by IPSS. We assessed (i) histopathological findings, (ii) early and long-term remission rates, and (iii) rates of additional pituitary hormone insufficiency. RESULTS Ten women and three men met the inclusion criteria. At 3 months, remission was noted in six (46%) patients: three (23%) had histologically confirmed adenomas, two (15%) had ACTH hyperplasia, and one patient (8%) was positive for Crooke's hyaline degeneration. New pituitary hormone deficits were noted in two patients (15%). After a median (±SD) follow-up of 14±4 years, recurrence was noted in two (15%) patients. Long-term control of hypercortisolism was attained by 10 patients (77%), with additional therapies required in nine (69%) of them. CONCLUSIONS In CD patients with failed intraoperative adenoma visualization, lateral one-third gland resection resulted in low morbidity and long-term remission in 31% of patients without the need for additional therapies. Bearing in mind the sample size of this audit, the indication for lateral one-third-gland resection has to be critically appraised and discussed with the patients before surgery

Persistent bone impairment despite long-term control of hyperprolactinemia and hypogonadism in men and women with prolactinomas.

While prolactinoma patients have high bone turnover, current data are inconclusive when it comes to determining whether correction of hyperprolactinemia and associated hypogandism improves osteodensitometric data in men and women over the long term. In a large cohort of including 40 men and 60 women, we studied the long-term impact of prolactinoma treatment on bone mineral density (BMD) in men versus women, assessed adverse effects of a primary surgical or medical approach, and evaluated data for risk factors for impaired BMD at last follow-up using multivariate regression analyses. Median duration of follow-up was 79 months (range 13-408 months). Our data indicate that the prevalence of impaired BMD remained significantly higher in men (37%) than in women (7%, p < 0.001), despite the fact that hyperprolactinemia and hypogonadism are under control in the majority of men. We found that persistent hyperprolactinemia and male sex were independent risk factors for long-term bone impairment. Currently, osteoporosis prevention and treatment focus primarily on women, yet special attention to bone loss in men with prolactinomas is advised. Bone impairment as "end organ" reflects the full range of the disease and could become a surrogate marker for the severity of long-lasting hyperprolactinemia and associated hypogonadism

Selective inferior petrosal sinus sampling without venous outflow diversion in the detection of a pituitary adenoma in Cushing's syndrome

Introduction: Conventional MRI may still be an inaccurate method for the non-invasive detection of a microadenoma in adrenocorticotropin (ACTH)-dependent Cushing's syndrome (CS). Bilateral inferior petrosal sinus sampling (BIPSS) with ovine corticotropin-releasing hormone (oCRH) stimulation is an invasive, but accurate, intervention in the diagnostic armamentarium surrounding CS. Until now, there is a continuous controversial debate regarding lateralization data in detecting a microadenoma. Using BIPSS, we evaluated whether a highly selective placement of microcatheters without diversion of venous outflow might improve detection of pituitary microadenoma. Methods: We performed BIPSS in 23 patients that met clinical and biochemical criteria of CS and with equivocal MRI findings. For BIPSS, the femoral veins were catheterized bilaterally with a 6-F catheter and the inferior petrosal sinus bilaterally with a 2.7-F microcatheter. A third catheter was placed in the right femoral vein. Blood samples were collected from each catheter to determine ACTH blood concentration before and after oCRH stimulation. Results: In 21 patients, a central-to-peripheral ACTH gradient was found and the affected side determined. In 18 of 20 patients where transsphenoidal partial hypophysectomy was performed based on BIPSS findings, microadenoma was histologically confirmed. BIPSS had a sensitivity of 94% and a specificity of 67% after oCRH stimulation in detecting a microadenoma. Correct localization of the adenoma was achieved in all Cushing's disease patients. Conclusion: BIPSS remains the gold standard in the detection of a microadenoma in CS. Our findings show that the selective placement of microcatheters without venous outflow diversion might further enhance better recognition to localize the pituitary tumo

Additional malignancies in patients with neuroendocrine tumours: analysis of the SwissNET registry.

PRINCIPLES Neuroendocrine neoplasms (NENs) are believed to be associated with an increased risk for additional malignancies (AMs). We aimed to (1) assess the occurrence of AM in NEN patients (2) investigate the characteristics and temporal relationship of NEN patients with and without AM. METHODS The SwissNET registry has prospectively documented patients with NEN since 2008, covering the entire area of Switzerland. Clinical characteristics, functionality, location and histology of NEN as well as survival of all consecutive patients were retrieved. The characteristics of the AM (location, histology, time point of diagnosis in relation to diagnosis of NEN) were extracted. RESULTS Out of 934 patients, 193 patients (21%) presented with AMs. There was no statistically significant difference with regard to location, functionality and grading (G1-G3) between the NEN patients with and without AM. AMs were diagnosed synchronously (±3 months), before (>-3 months) and after (>+3 months) diagnosis of NEN in 82 (42%), 96 (50%) and 13 (7%) patients, respectively. Location of NEN correlated with the anatomical origin of the AM. Age- and gender- corrected survival was not significantly different between NEN patients with and without AM. CONCLUSION The prevalence of AM in NEN is high. The comparable characteristics with regard to functionality and grading in the NEN cohorts with and without AM and the similar location of AM and NEN suggest a selection bias towards frequent imaging procedures in NEN patients with AM

Machine Learning for Outcome Prediction in First-Line Surgery of Prolactinomas.

Background First-line surgery for prolactinomas has gained increasing acceptance, but the indication still remains controversial. Thus, accurate prediction of unfavorable outcomes after upfront surgery in prolactinoma patients is critical for the triage of therapy and for interdisciplinary decision-making. Objective To evaluate whether contemporary machine learning (ML) methods can facilitate this crucial prediction task in a large cohort of prolactinoma patients with first-line surgery, we investigated the performance of various classes of supervised classification algorithms. The primary endpoint was ML-applied risk prediction of long-term dopamine agonist (DA) dependency. The secondary outcome was the prediction of the early and long-term control of hyperprolactinemia. Methods By jointly examining two independent performance metrics - the area under the receiver operating characteristic (AUROC) and the Matthews correlation coefficient (MCC) - in combination with a stacked super learner, we present a novel perspective on how to assess and compare the discrimination capacity of a set of binary classifiers. Results We demonstrate that for upfront surgery in prolactinoma patients there are not a one-algorithm-fits-all solution in outcome prediction: different algorithms perform best for different time points and different outcomes parameters. In addition, ML classifiers outperform logistic regression in both performance metrics in our cohort when predicting the primary outcome at long-term follow-up and secondary outcome at early follow-up, thus provide an added benefit in risk prediction modeling. In such a setting, the stacking framework of combining the predictions of individual base learners in a so-called super learner offers great potential: the super learner exhibits very good prediction skill for the primary outcome (AUROC: mean 0.9, 95% CI: 0.92 - 1.00; MCC: 0.85, 95% CI: 0.60 - 1.00). In contrast, predicting control of hyperprolactinemia is challenging, in particular in terms of early follow-up (AUROC: 0.69, 95% CI: 0.50 - 0.83) vs. long-term follow-up (AUROC: 0.80, 95% CI: 0.58 - 0.97). It is of clinical importance that baseline prolactin levels are by far the most important outcome predictor at early follow-up, whereas remissions at 30 days dominate the ML prediction skill for DA-dependency over the long-term. Conclusions This study highlights the performance benefits of combining a diverse set of classification algorithms to predict the outcome of first-line surgery in prolactinoma patients. We demonstrate the added benefit of considering two performance metrics jointly to assess the discrimination capacity of a diverse set of classifiers